Discussion of “ Analysis of Variance — Why It Is More Important than Ever ” by a . Gelman

Bayesian inference and fixed and random effects. Professor Gelman writes " Bayesians see analysis of variance as an inflexible classical method. " He adopts a hierarchical Bayesian framework to " identify ANOVA with the structuring of parameters into batches. " In this framework he sidesteps " the overloaded terms fixed and random " and defines effects " as constant if they are identical for all groups in a population and varying if they are allowed to differ from group to group. " Applying this approach to his first example (a Latin square with five treatments randomized to a 5 × 5 array of plots), variance components have to be estimated for row, column and treatment effects. In our opinion, his approach provides an insightful connection between analysis of variance and hierarchical modeling. It renders an informative and easy to interpret display of variance components that is a nice alternative for traditional analysis of variance. However, we wonder whether sidestepping the terms fixed and random is always wise. Furthermore, currently his approach is rather descriptive, and does not contain truly Bayesian inference. Both points will be briefly discussed in the sequel. To look into the question of fixed versus random and the use of hierarchical modeling, we carried out a small experiment. We constructed a dataset for the example in Section 2.2.2: 20 machines randomly divided into four treatment groups, with six outcome measures for each machine. We asked a statistician who is very skilled in multilevel analysis to analyze these data. The result was a hierarchical multivariate data structure with six outcomes nested within 20 machines, and the treatments coded as dummy variables at the machine level. Variance components were estimated for machines and measures. The treatment effects were tested by constraining all treatments to be equal and using a likelihood-ratio test. Comparing this procedure with the discussion of this example in Gelman's paper shows that this is not what he had in mind. It certainly contradicts